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ABSTRACT 

Methodologies and trends of judicial behavior k 
research* (the study of how knd why court .judges make decisions) *are 
traced from the 1920s to the present. Arranged into five sections, 
the first two sections of the report identify principal points of 
social science research in general. A summary of methodology in 
judicial behavior research reveals that the majority of researche,rs 
use, judges' decisions as their primary data source; however, this 
source is being increasingly supplemented by comparative analyses of. 
judges from American and non-American systems. The third section 
reviews research during. the "pioneer days" from the "Ml20s to th^ 
publication of Schubert's "The Judicial Mind" in 1965. The 
introduction of factor analysis, multiple regression, bloc, -and 
scalogram analyses are noted as the most significant statistical 
developments of the period. The fourth section traces methodology 
from "The Judical Mind" to the present. The psychometric model, 
comparative analyses, enhancement of methodological foundations, 
expansion of survey research, content analysis, a^nd longitudinal 
studies are characteristic o± this period. The last section assesses 
the state of the art and suggests that comparative studies and 
experimental designs /will increase and more effective use of known 
statistical methods will occur. (KC)' 
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THE DEVELOPMENT OF THE METHODOLOGY 
OF JUDICIAL BEHAVIOR RESEARCH: A HISTORICAL REVIEW AND CRITIQUE 
OF THE USE AND TEACHING OF METHODS 



I. Introduction; The Meanings- of Methodology and Judicial 
Behavior 

•» 

/ 

It is necessary to begin by identifying the principal 
subjects to be discussed: methpdology and judicial behavior. 
The two terms can be subject to so many interpretations as to 
make a discussion of their conjunction in political research 
unmanageable. 



Methodology 

Casual social scientific use of the term methodology is 
likely to equate it with the use of a statistical method, or even 
mpre narrowly, with the application of statistical measures to 
quantified data. But methodology is a broader subject,' indeed 
one which cannot be easily Separated from "theory building." For 
the purpose of separating theory from methodolpgy^ the - latter 
will here be considered to encompass frve components: research 
design and comparative method, data generation procedures, opera- 
tionalization and measurement of concepts, statistical methods 
(techniques), and statistical measures. 

Judicial Behavior ^ • 

"Judicial behavior is part of the broader approach in 
•political science that has come to be called behavioralism. : . . 
•[Wlhat distinguishes judicial behavior from behavioralism 
generally is its specific focus upon.^he decision making of 
^ judges" (Schubert, 1968: 307). Alternatively, the "judicial 
behavior approach represents the fusion of theories and methods 
developed in the various social sciences. . . in order to study 
scientifically how and why judges make the decisions they do" 
(Sphubert, 1964: 3). These definitions f tom T;he scholar who most 
xieserves to be called the father of j-udicial beha*vioralism 
delimit th#^ focus of this article . , Nevertheless^ the term 
"judicial- decision making" will not^be interpreted especially 
narrowly. ^ The formally rendered judicial decision represents the 
end of the judicial decision making process. The beginning of 
that process occurs when the judges are recruited to pf JEice, and 
there is a variety of intervening processes which shape that 
formal decision. ^ ' - ^ ^ - 

To make the concentration of this arti<:le clear, it may Be 
worthwhile to indicate what sort of "judicial" studies are not 
included within ^ its purview. .The first excluded category is 



traditional dbctrinal analysis, which does not fail within the 
boundaries of Schubert *s , definition of judicial behavior. Also 
excluded ^ are studies of the impact or policy consequences of 
judicial 'decisions, however important these may be. Finally, 
studies which focus on the behavior of actors relateji to the 
judicial process who are not ^ judges — litigating interest 
groups, executives who appoint judges/ electorates who elect 
them, elite and mass publics who evaluate their decision making 
— are excluded. Al^ these kinds of studies may be important to 
one who wishes a full understanding of the judicial process, bxity^ 
none focuses on^ the decision making which is at the heart of 
judicial behavior; 

ll'. Aspects of Methodology 

♦ 

Research Design and Comparative Method 

The modal research design of judicial behavior analyses is 
post hoc and implicit, i.e., an artifact of available or chosen 
data generation procedure^ or even of knowledge or choice of a 
statistical method or measure. Thus it will not be fruitful to 
present a full discussion of research design in the sense in 
which that term has come to be used 'as a result of the influence 
of Donald Campbell and* his*' associates (see Campbell and Stanley, 
*1963 and the 'discussion in Leege and Francis, 1974v Ch. 3 and the 
references cited therein). In di9<:ussing research design, then, 
I shall (following Dijphart, 1971) focus on whether a study 
employs a "compar;ative', " or "case study" methoa, instead of or 
in supplement to the standard "statistical" method, realizing 
that, there are, so far, no examples of true "experimental" 
judicial behavior studies . Special attention will be given to 
studies which compare two or more occurences of the phenomenon 
under investigation, either explicitly or by following the 
example of or incorporat'ing direct references to the findings of 
studies on other occup;^aaes of the phenomenon. The reason for 
this is simple: «5VriV by increasing the scope of the 
generalizations drawn fro?ti judicial belta^vior research will middle 
and broad range theojbi?es of judicial! behavior 'be' achieved. 
Attention will be given , t6 whether designs are* cross-sectional or 
longtitudinal , and how the design might have been improved. 

The "unit of analysis," another important topic in research' 
design, will be a matter of minimal discussion he^re because the 
definition* of judicial behavior to be used will limit analytical 
attention to studies either employing individual judges as units 
of analysis or,, in a few cases, judidial decisions as units 
allowing generalisations about the bel}avior of individual judges. 

Data Sources and Data Generation Procedures 



The modal gudicial behavior study uses data drawn from 



official records^'such as court reports^or quasi-of f ic}.al sources 
such as the reference series published by law publishers. ^ This 
category of data is complex — it incorporates a variety of types 
of information — and rarely self -representing in any meaningful 
sense: it almost always requires considerable "coding" (of votes, 
outcomes, policy or legal issues, etc.) by sophisticated 
investigator before it is useful for research on judicial 
behavior. To conduct the required coding, the investigator 
ordinarily must engag'e in some kind of content analysis , even if 
relatively superficially. More intensive content analyses of 
official and unofficial documents have been relatively rare, but 
of substantial importance in the development of theory in'' some ^ 
areas of judicial behavior. Judicial behavior research has also* 
been based on data generated by tj^e other common methods of 
social science data generation, including survey research using 
mail questionnaires or personal interviews , field observation , 
and, rarely, human ,or computer simulation . Although much 
discussed by some legal scholars and judicial' behavioralists , 
clinical examinations — psychological and/or physiological — 
have as yet been used .even more rarely to produce data relevant 
to explaining judicial behavior. 

Operationalization and Measurement 

Operationalization and measurement ^ of the concepts from 
which theories are built have received less attention in judicial 
behavior research than they have deserved. ' Unavoidable 
weaknesses in available data may have led some analysts to 
conclude tijat little could be done to reduce the error variance 
which impede^ the verification of plausible hypotheses. The 
"mechanical" application of statistical methods made possible, if 
,not^^ defensible, by "user-friendly" modern computer programs may 
have diverted the attention of others from that beginning adage 
of computer analysis: "garbage in, garbage out." And, since 
operationalization and measuremeat is not likely to be of . much 
.concern in' the absence of a 4:'heory which imbeds concepts 
requiring operationalization and measurement, it is also. cer±ain* 
that inadequate theorizing has led to weak operationalization and 
measurement. 

Good measurement is hard work. Establishing the reliability 
and validity of indicators can be more difficult than conducting 
the test of the hypotheses the indicators have been contructed to 
operationalize. Thus • I would not wish to endorse a "purist" 
position which rejected any analysis using measures whose 
reliability and/or validity could not be unequivocally 
demonstrated. In many cases, face validity will have to -suffice 
and resources will' prevent full investigation of the reliability^ 
of the indi^cators employed. But it will be helpful in reviewing 
judicial behavior research to assess the extent to which measure- 
ment practices have affected potential substantive conclusion^. 
Special attention will be paid to the use [of multiple indicators, 
the construction of scales and indices which, in principle , ' allcjv 
more robust measurement of a concept, and to the consideration of 



the ppssible interactions between variables which might have a 
significant statistical impact and an appealing theoretical 
interpretation. ^ ' ' . 



Statistical Methods 

I distinguish between" " statistical metHods" ("techniques") 
and •'statistical measures .'V This i^s less arl>itrary than it might 
at first appear. Statistical methods aVe the broad approaches to 
the presentation and analysis of data which have been developed 
to help answer important classes of questions^ one asks as an 
investigator. Answering these, questions is . crucial to 
determining whether and to what ^extent research hypothese;s are ^ 
verified. Using the wrong method will make it difficult, if not 
impossible, to determiae • the fates of one's hypotheses, U^ing 
the wrong^ measure is less likely 'to occur' if one is using the 
right method, but is also less likely to misle^d-'tlie analyst as 
to the fate of his/her hypotheses if it does occur. The family 
of statistical metlibds includes univariate analysis of , central 
tendency, dispersion, and distribution, classical contingency 
table analyses using some variation of Lazarsf eld's "elaboration 
model" (see Rosenberg, 1968), analysis of variance, linear and 
nonlinear regression analysis (simple and multiple), factor 
analysis, uni- and multi-dimensional scaling, causal modelling 
including path analysis (recursive and nonrecursive) / time series 
analysis— (Including ARIMA modelling of o-nterrupted time series) 
and a host of, in my view, ordinarily less useful techniques- 
generally derived from one or more of the preceeding, including 
"tree" or AID analysis and ' its" analogues for categorical 
dependent variables such as THAID, discriminant function 
analysis, canonical correlation, bloc and cluster analysis, and 
log linear modelling and related methods for the analysis orf 
categorical variables in contingency tables. " ^ 

Statistical Measures ^ 

Statistical measures are the; specific coefficients . 
calculated in the process of employing statistical methods. They 
are classically divided into descriptive and inferential 
measures, b\?t beyond univariate measurement, they pan be more 
meaningfully^ classified as measures of association and measures 
of statistical significance. The former are far more important 
than the latter for theory building, for they tell an 
investigator how. strong is (are) the relationship(s) depicted by 
his/her hypotheses. Examples of measures of association include 
lambda, phi, gamma, and the family of Kendall's and Goodman and 
KruskaT's taus for contingency tables, the simple and multiple 
correlation coefficients and their squares (r, r^, R,. R^), and 
the unstandardized and standardized regression cdef f icients (b's 
or betas and. "beta weights"). Measures of signific^ce cannot be 
dispensed with, hoWeve;tj , since they tell a'n^ invesltigator the 
probability that a relationship as strong as tha^ discovered 
could have occured by chance, thereby establishing a ''"level of 



confidence" for the relationship. Examples ef measures of 
significance include Chi Square, Piab^i;'s Exact Test, the Mann- 
Whitney U test, and the z, anp F coefficients. Judicial 
-behavior- scholars have not always used the ' most appropriate 
statistical measures- for the data and statistical methods 
characterizing their stoidies. * - 

A Suxmnary Assessment of Methodology in Judicial Behavior Research 

Recent work by Hensley with his students (1981a, 1981b) has 
provided a useful summary of a large and probably representative 
body of judicial research ac;:oss the decades pf the 1960 's and 
1970 's. They content anpilyzed every article dealing with courts, 
judges, 'and the judicial process which cippeared in four major 
political science journals from 1961jl980 and reported summary 
data usually arrayed over tirae for such study characteristics as 
data base(s), quantification?, statistical techiiiiques, court(s) 
studied, substantive focus, and theoretical /framework used. 
/^hese data illustrate the general patterns o*f development of many 
aspects of methodqlogy in judicial behavior research. 

Hensley's data reveal a steady increase in the use of 
quantification in judicial articles across the period 1961-1980. 
For tjie whole period, 58% of the 223 articles^published. in the 
surveyed journals used at least minimal quantirication (defined 
by. the presence of one or more numerical tables). The 
quantitative proportion rose at first sharply then more gradually 
from 36% in 1961-1965, the period of the firm establishment of 
trie- judicial behavior approach, to 72% in 1976-1980. While not 
all these quantitative articles ^ would qualify as judicial 
' behavior studies, it is apparent from a review of their titles 
that a large portion would, and it is also likely that, those 
which wouLd not would-* differ only marginally from the judicial 
behavior articles; Thus Hensley' s discussion of the quantified 
articles in his sample should provide a reasonably accurate 
assessment of the development of judicial behavior research. 

Data Sources 

Hensley' s itiarginals for the data sources used by the 
quantitative studies in his sample give an overall picture of the 
data sources of judicial behavior research fo^ the las^ two 
decades: - 

Data Source Used % Using 



Judges. Decisions (votes, sentences,' 65% 
opj^rrrtDns ) ^ 

Case information from court documents 11* 
other than judges decisions 



i 



Background information on judges from 22 
non-survey sources 

Mail survey 9f judges , 3 * 

Personal interview survey of judges 7 

Survey research of Aon-judges " 11 ' ' . 

Demographic data 12 

Other sources ^ \ 28 

ise data clearly show' the domination of judicial behavior 
r'*^earch from 1961-1980 by that most traditional of public law 
data sources, judges decisions. This domination is even, more 
impressive if one realizes that ^studies using sur'vey data from 
non- judges and demographic data are not very likely to be 
judicial behavior studies , as that term is defined here. 

Hensley also provides a cross-tabulation of these data 
sources over time. That display (his Table 5) shows very little 
variance in the proportion of the quantitative studies^ using 
judicial decisions^ as a data source, although the 1961-1965 
period percentage \is * higher than the rest at^ 85% of the 20 
articles published \ in that period. The most consistent and 
substantial increases occur in the personal interviews of judges 
and "other" categories. By 1976-1980, personal interviews of 
judges were used ia 10% of the studies while^pther source of data 
were used in 46%. Certainly these trends are signs that the 
variety of data sources used in judicial behavior r^earch has 
increased, despite the continued importance of the staple data 
source, ) judicial decisions, votes, ^nd opinions. ^ 

Statistical Methods and Measures ^ 

For specific quantitative methods. used, Hensley reports, the 
following overall results for the 1^30 quantitative articles: 

Method/Measure Used %^ Using 

Univariate/bivariate descriptive , 88% • 
statistics at nominal/ordinal level 



Univariate/bivariate inductive ' 35 
statistics at nominal/ordinal level 

Univariate/biyariate descriptive 35 
statistics at interval level 



Ohivariateybiv^riate inductive 
statistics at interval le\^el 



15 



Multivariate descriptive 9 
statistics at nominal/ordinal level 



6 



26 

Multivariate inductive 15 
statistics, at ijj|^rval level 

Bloc Voting AnaLysis 9 ' 

Sea log ram Analysis 19 V 

It is unfortunate that Hensley's coding scheme did not 
differentiate between the use of univariate and bivapiate 
methods, since the use of the latter implies the existence of the 
relational analysis necessary to hypothesis testing and 
explanatory research^ But even without this distinction, his 
data might be taken to indicate that the level of statistical 
sophistication of judicial behavior research has not been not 
very great, given the percentages 'for the use of various types of 
multivariate mefc^iods. An over time breakdown for the use of 
multivariate , methods shows that for the first fifteen years of 
Hensley's journal survey, there was no trend'«^toward any increase 
in their us^:\ only in the 1976-1980 period did the percentage of 
.quantitative articles using multivar^-ate methods increase f r:om 
"just over 20% to 46%. For the twenty year period, the proportion 
using any multivariate method was only 31%. 

The use of bloc analysis and cumulative scaling, two classic 
methods of dimensional an^ilysigt in judicial behavior research, * 
declined over the ^twenty years surveyed. For the former>. the 
decline was from 15% in the I960 ' s studies to only 4% in studies 
published fr6m 1976-t.l9!8,0. For scalogram analysis, the decline 
although uneven^ wa?? from 35% in 1961-65 to 12% in ^ 1976-1980. 
These declines couLd^mean that the methodological sophistication 
developed by the pioneers of judicial behavior has disappeared in 
the work of their successors. Somewhat more optimistically, it 
may mean that these' classic techniques were superseded by^ more 
powerful multivariate methods, at least an the last half of the. 
1970's. My impression is that it' also reflects a broadening of' 
the substantive focus of "judicial behavior research to areas in 
which bloc , and scalogram analysis are^less appropriate. 



Research Design 

The fact that Henslpy's studies did not specifically include 
a classification for study research design is almost certainly 
not the fault of ^ the researcher, but is an indication -Of the lack 
of explicit attention received by this^ topic in judicial behavior 
research. In the*absence of systematic data on study designs, an 



Multivariate inductive 

statistics at nominal/ordinal rLevel 

Multivariate descriptive 
statistics at interval level • ^ 



impressionistic . assessment of research design * patterns in 
judicial behavior research (supplemental to that given , above) 
will have to suffice, ^ 

It appears that relatively few judicial behavior studies 
should be classified as "case studies," at least in the sens6 in 
which* this term is used by Campbell and Stanley (i.e*, "the one 
shot case stiidy" ) . Judicial "behavior scholars h^ve recognized 
the limitations of the case study: after all, the judicial 
behavior movement represented a revolt against the case study 
method as used in traditional public law. On the other hand, 
there have be6n instances of what Lijphart (1971: •691-92) calls 
"hypothesis generating, theory confirming, theory infirming, and 
deviant \case ^fetudies, " which he argues may Jiave great value for 
the development of political theory. Outstanding examples would 
be 'Danelski*s (1^64) detailed analysis of the. appointment of 
Pierce Butler to the U.S. Supreme Court , and Schub,ert*s (1965a) 
analysis of the values of ♦Justice Jackson. Parenthetically, the 
latte'r is an excellent indication that case studies ♦ neej^d not be 
quantitatively unsophisticated. 

i At the other end of Lijphart !s methodological continuum, the ; 
central focus of judicial behavior research on judicial decision * 
making has apparently precluded any experimental research. 
Instead, the method' of choice (necessity?) has .been 
"statistical," supplemented increasingly , by the introduction of 
the "comparative" method as data on a greatet variety o^ corps of 
• judges from different American and non-American political 
systems. ^ ^ - ^-.^ * 

As coded, Hensley's- data^, do* not allow a' definite 
determinatrc^n of the extent to which^ judicial behavior studies 
have rntrodi:^ced some version of the comparative method into their 
analyses. lNevert|:ieless , his coding of the - "courts s'tudied" 
details somlp categories which are prima facie "comparative." 
Thus Hensley reports that 16 studies dealt with more than one 
level of th^ federal courtis and 3 with more than one state court 
lev^l , 11 dealt with both federal and state courts,. 9 dealt with 
nonr^American courts, and 2^ with international courts. This means 
Jbhat 41 (42%) of the 97 studies (69 of which were quantitative) 
which did not deal only with the U.S. Supreme Court can be 
verified to have had at least an implicit comparative, focus. It 
is clear that many of the other 55* studies in this category also 
dealt with multiple courts, even though Hensley' s coding does not 
allow one to calculate how many. Outside the study of the U..S. 
Supreme Court, the^ use of th^ "comparative method," often in 
conjunction with the "statistical method/" has be^n more 
•significant than one migiit have expected. Hensley 's data also 
suggest that comparative designs became more frequent over the 
two decades surveyed. 

Henile^^^ fQur^d 61 quantitative studies of the U.S. Supreme 
Court on]3S^^ "^^Ithough fhis paper does not present any breakdown to 
indicate Phe extent to which these ^studies are cross-sectional or 
at leastrpartially longitudinal in their designs, it is clear 



that many of the 63, articles, present cross sectional analyses rOf 
the behavior of Supreme Court ' justices at; more than one point fn 
time. To the. extent that this, is true, even these studies have 
some V comparative component, • . ^ ^ \ 

\ 

More^ generally,,, most judicial behavior research has been 
cross-sectional qr\ at best, Has analyzed successive crbss- 
sections at irtultip^e points in tipie. Traditional "time series" 
studies,, which analyze 'the 'behavior of a single unit, of analysis 
over time have .been more/characteristic,of judicial studies using 
the court, rather th^n* judges; as a unit of analysis. 
Nevertheless, recent work by Ulmer (1979a, i979b, 1973a) 
exemplifies true longitudinal research using individual justices 
as units of Analysis:^ 

Operationajization and Measurement 

Except to the extent that it details the use of bloc 
analysis,^ which has been used to operationalize cbnflict . and 
consensus in -judicial decision making, 'and culnulative scaling^ 
whiph has been used to operationalize* judicial attitudes and 
values, Hensley's research has little to say about patterns of 
Qperatio^nalization and measurement in judicial behavior research, 
Ift a preceeding paragraph, I indicated that insufficient 
attention j ha<J been given to these matters . in most judicial 
•behavior research. Yet in its use of and cluste];: analysis, "power 
indexes," and aniumb^r of ad " hoc measures, judicial behavior 
research has devoted- substantial^ attention to operat:ionalization 
in sdme arpas of inquiry. It is not accidental that the **most 
satisf actol^y theoretical work has been produced in the areas in 
which^ bperationalization and measurement have been taken most 
seriously^. " ' 

III, Beginnings: . jyiejthodology in J:he Pioneering Studies of 
Judicial Behavior 

' ' ' . ^ - 
I would date the "pioneer days" of judicial behavior 
research as lasting until the publication of Schubert's The 
* judicial Mind in 19.65, From the 192.0 's until its publication, 
there had been first isolated attempts to deal with the decision 
ihaking of judges through the use of systematic quantitative data, 
then agressively self-conscious efforts to begin the integration 
of the study, of public law into the behavioral movement and to 
battle the bastions of traditionalism. Efforts to develop 
appropriate methodologies for judicial behavior research had made 
great progress, 

Ttie Judicial Mind marked the end of the pioneering and the 
beginning of the modern era because it represented the first full 
scale, completety behavioral, metbd^o^ogically sophisticated 
effort to develop a theory of JJud^Tcial decision making in the 
world's most celebrated court , It represented a culmination for 
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^the efforts of such important judicial behavior pioneers as Mott 
al^, Gaudet, Pritchett, Thurstone and Degan, Snyder, Ulmer, 
Nagel, Spaeth, Schmidhauser , Tanenhaus, Danelski, and Schubert 
himself. It also served as a reference point -for later 
inquiries. 



From Pre-Behavioral Stirrings to Behavioral Self^-Consciousness 
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discussing the predecessors of The Judicial Mind 
behavioralism ia general, it Is customary 
s The fioosevelt Court (1948) as the most significant 
work inspiring the development of the judicial behavior movement. 
Methodologically, however. The Roosevelt Court must share its 
significance .with earlier quantitative jiTdicial studies 
initiating research traditions which were important in the 
development of judicial behavior,' and with Schubert's 
Quantitative Analysis of Judicial Behavior (1959). The latter 
surveyed and sytematized the presentation of a family of 
quantitative approaches to judicial behavior which shaped the 
development of the°field, and whigh are still effectively' used , 
.despite the availability of sometimes more appropriate modern 
methods of which Schubert himself has been a leading proponent. 
The pubication of Quantitative Analysis of Judicial Behavior 
serves as an appropriate demarcation for the . beginning of a 
period of self-conscious attenticm to judicial behavior by a 
significant number of creative scholars. 



Bibliographers of judicial behavior have pointed out a 
number of studies published as early as 1919 which used data and 
at least rudimentary quantitative analysis to investigate the 
behavior of judges (see especially Vines, 1970; Schubert, 1964). 
The majority of these early studies (Everson, 1919; Gaudet, 
Harris, and St. John, 1933, 1934; Gaudet, 1938, 1946; and the ^ 
studies cited in Green, 1961: -1-28 ) focused on Variations in 
'sentencing behavior, ., a topic which continues to be intensely , 
studied today. All were based on data drawn from court records. ! 
and subjected mostly to simple univariate and bivariate" analyses , 
as was appropriate given the development of social research in* 
their day. What was most^ striking about the Everson study "was 
the very large sample on which it was based: more than 150,000 
cases from New York City. Gaudet 's studies were based on more 
conventionally sized samples, but exhibited considerably more 
sophistication in data- analysis : the most extensive (Gaudet, 
1938 ) presents carefully detailed tables displ'aying the variation 
in. judicial sentencing by type of crim^ and over time. 
Nevertheless, the most elaborate statistic us/ed is the percentage 
and there is little effort to engage in the systemeitic tabular 
analysis encapsulated in the e-laboration modtel, 



The sentencing studies cited above wer^ the only attempts to 
analyze individual judicial decision making quantitatively prior 
to Pritchett' s work which- resulted in The Roosevelt Court. There 
were other studies which presaged later developments in judicial 
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Mott, Albright, and Semerling ( 1933 ) , who studied the personnel 
of 32 " state supreme , courts plijs all federal judges 'sitting 
'beliween 1900 and 1932 .(N=685), Ewing^ (193^), who discussed 

only- ^ Supreme Court" justices serving through 'the Hoover 
^ administration. Both studies. were descriptive, presenting simple 
. tables • displaying the . frequency , distributions ' for various 
characteristics of their samples of jiudges. But .while Ewing 
- presented his data as .an adjunct to moire traditional historical 
.and normative ankl^^is Mott .et al • demonstrated substantial 
sensitivity to me tho<k) logical considerations I For example/ they 
carefully descri^ed(l their sample and dat^ source ( Who ' s Whol , 
discussedn the-dat-a^'s validity, engaged in the construction ok 
elaborate indexes to measlire sta*t;e supreme court prestige (based 
on a survey of law school professors), quality of the judicial 
personnel, and '.citations by other state supreme courts and the 
, Supreitte Court, correlated the perspnnel. index ,^ith the 

^ pther^d (to investigate its validity), reported confidence 
■ inteifvals for th,e correlation coefficients, analyzed variation in 
*. the personnel quality index by state,, type of court, and method 
of selection, and engaged in rudimentary longtitudinal analysis 
of ^^I5t^eral judicial characteristics to determine "which, way is 
-the wind^ blowing?'." Although the study did not present any 
^multivariate analysis beyond twp way analysis of means, a 
forgiveab^le flaw given its dat-e- of publication, it could 
otherwise serve as a model for contemporary analyses of similar 
topics/ ^ ' ' 

. There is little doubt that The Roosevislt Court is the most 
important precursor of behavioral efforts to study judicial 
decision making in the. Supreme Court, Methodolbgically , 
.•Pritchett 's use • of cluster bloc analysis of majority and 
dissenting opinions was extremely influential* Hensley's (1981b: 
55) d^ta indicate that, as' refined by^Schubert (1959) and Sprague 
(1968), it continued to be used in quantitative judicial analyses 
into the 1980's. Pritchett ' s operationaliz.ation of bloc 

• membership as indicative of shared liberal and ^ conservative 
-judicial values and his tabulations of individual differences in 

those valuer in terms of -judicial voting percentages- pro and con 
claimed- political liBe^rtn.es and economic priveleges may be seen 
as 'the foundation of the psychometric model which has dominated 
analysis of Supreme Court justices* decision making. 

Pritchett^s analysis' in The Roosevelt Court was ' brilliantly 
/ntuitive.. His deep qualitative understanding of the Supreme 
Court allowed him to to produce this influential work while 
devoting no attention to problems of research design, being 
fairly casual about defining his sampling universe and unit of 
analysis (see Schubert, 1959: 164-66), ^©jS^nexplUcit concerning 
the methods used to ord,er his bloc matribes. As a consequence, 
it would be difficult for less talented analysts to duplicate his 
effort with good results, . ahd it is^ likely that even an equally 
tal.ented analyst might come to slightly different conclusions in 

* ^n effort to replicate Pritchett's analysis, 

, ^ Pritchett alao made ^ no effort to use any quantitative 



techniques beyond the calc?ulation of percentages of agreement and 
favoring various policy claims. His work makefe no reference to 
the efforts to make cluster bloc analysis, more systematic and 
powerful .by Rice (1928) and by his colleagues in .other 
disciplines at the University of Chicago (Beyle, 1^31; Thurstone, 
1932). Possibly for these, as well as other reasons, it was not 
until Schubert specifically addressed the* methodology of bloc 
analysis in judicial behavior sthat Pritchett's techniques came 
into 9ommon use. 

Schubert's Quantitative Analysis of - Judicial . Behavior was 
the methodological clarion^ call for the judicial behavior 
'movement. In it he illustr^ated the application of four 
analytical approaches which have been extremely important in the 
development of judicial behavior research.' One of these ("The 
Analysis "of Summary Judicial Power") uses the court as unit of 
analysis and is .thus not of direct relevance hei/e, even* though' it 
introduces analytical themes • which are of importance in- the 
judicial politics literature more generally. The other 
approaches are'^'^^bloc analysis,, game analysis, and scalogram 
analysis. All three ^ remain important approaches- to judicial 
behavior research eve*n though Schubert abandoned simple bloc 
analysis in favor of more powerful dimensioning techniques in his 
own later research. 

Given its ^central concerns, it is not surprising that 
Quantitative Analysis of Judicial Behavior devoted no conscious 
attention to research design or data sources in judicial behavior 
research. ' Jt is essentially ^ "how to" manual which uses 
significant data to illustrate the methods it advocates'. , It 
does, however, ^ pay attention to operationalization and 
measurement within the domains it explores.^ For example, in 
discussing bloc and scalogram analysis, Schubert - is quite 
specific about. the data and the,units*of analysis to be used. He 
also makes clear* that he sees scaiogram analysis^ as a better 
means of operationalizing thdt judicial values which Pritchett had 
summarized by voting percentages, and introduces various indexes 
^ of agreement as maans of more systematically assessing the unity 
and levels of conflict in blocs and courts. 

The principal methodological problems of the techniques 
demonstrated in Quantitative Analysis of Judicial Behavior rested 
in the indeterminacy which still remained in * Schubert's 
refinement of Pritchett's bloc analysis procedures and, to a 
lesser degree, in the scalogram construction procedures it 
outl^es. Although Schubert tried to formalize bloc construction 
proc^i^ures as much as pf5^ssible, he noted that bloc analysis must 
still proceed by trial and error, since he had been unable to 
come up with "an unfailing routine (1959: 83)." As an 
alternative to bloc analysis in the Pritchett tradition, he 
proposed McQuitty's Hierarchical Syndrome Analysis, a primitive 
form of hierarchical clustering analysis (1959: 167-72), ajid 
suggested a refinement "to th-e procedure by establishing an 
agreement index which measured how much more or less ' a group 
clustered than chance expectations would have dictated. With 
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typical candor/ Schubert reported the problems with this 
desirable measure and his ihability to derive a method for 
putting confidence intervals around the index. 

The problems with Schubert^ s method of scalogram analysis 
were arguably less serious, and and were publicized in a critique 
by Tanenhaus (1966), the scholar who had first suggested the 
applicability of cumulative scaling to judicial decision making. 
They revolved around the^ indetermi nancy of the placement of 
judges on scalograms under k variety of case marginals conditions 
and the relative ease of constructing acceptable scalograms when 
jthe number of itfems greatly exceeded the number of ' persons 
responding to those items, the rule in judicial behavior reseai>fch 
but the clear exception in the mass behavior research settinos in 
which scalogram analysis had been developed. 

A solution to the problems with the bloc analysis procediires 
presented in Quantitative Analysis of Judicial Behavior became 
available when Sprague (1968) developed invariant procedures for 
bloc analysis. The development of "objective" procedures for 
cumulative scaling in computerized statistical packages reinoved 
some of the problems of indetermi nancy /in Schubert's suggested 
methods. Furthermore, Schubert's ownl later work in cumulative 
scaling proceeded by different rules which were jLess restrictive 
(see Schubert, ^1967b). Tanenhaus ' suggestion that scales should 
not count multiple cases which present the same voting stimulus 
has occasionally been heeded, but most^^ften has*^not. 



Self-Conscious Judicial Behavior Resear'ch 



Bloc analysis, cumulative scaling, and the game theory 
approach suggested in the fourth chapter of Quantitative Analysis 
of Judicial gehavior have each been associated with the 
development of a fecund research trend in judicial behavior. 
Articles reporting applications of each had begun to appear 
contemporaneously with the publication of Quantitative Analysis 
of Judicial Behavior , and appeared in great numbers as the 
methods it prescribed became more widely known. Bloc analyses 
was used to describe levels df conflict and consensus in multi- 
judge courts, which then were often explained by reference to 
other information. Scalogram analysis became the standard method 
for exploring or demonstrating the existence of policy attitude 
dimensions underlying and, at one level, explaining judicial 
voting behavior on such courts. Game theory inspired efforts to 
suggest and explore strategic group processes which might, at 
another level , explain judicial decision making, including 
decision making in circumstances other than those involved in the 
formal disposition of cases. Few if any of these studies made 
contributions , to the development of methodology in judicial 
behavior research beyoad those in Quantitative Analysis of 
Judicial Behavior . ^ Instead,^ these studies represented the 
emergence of a self-conscious sub-discipline. But there were 
other developments which were methodologically significant in 
these years of consciousness-raising.. 
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Among the most significant developments was the 're--emergerice 
of ^ research on judicial attributes and recruitment, which had 
lain dormant since the work of Ewing and Mott et al^ . discussed 
above, and of explicit attempts to take the next analytical step 
implied by the study of judicial attributes: to link attributes 
to judicial decision making/ Contributions to the data sources 
of judicial behavior research were made by Schmidhauser (1959), 
who systematized a large body of information conSerning the 
justices of the U.S- Supreme Court., Nagel (1962), who coded, 
biographical d|ta secured from directories on all ^ state and 
federal supreme court justices sitting in- 1955, Vines (1963), who 
reported data on the Louisiana state judiciary, and Torgerson 
(1963), who analyzed the backgrolands ,6f the Norwegian judiciary, 

Torgerson »s work was also i^ignificant in that it represented 
the first attempt published in English to provide a comparative 
dimension to the design of judicial behavi^or research. Also 
noteworthy in this regard was the work of Hayakawa (1962), which 
represented the first effort to apply 'cumulative scaling to the 
behavior of non-American judges • 

The effort to li^ik ^'judiqial attributes to judicial decisions 
was carried forward by Schmidhauser and Nagel by different means. 
The former (see Schmidhauser, 1964 for a useful summary) analyzed 
the relation between a variety of attributes and cumulative 
scales of "regionally divisive" cases in the nineteenth century 
and tendency to abandon stare decisis > The latter (see Nagel, 
1969 for a good summary) developed specialized ^measurement 
methods to allow him to cppe with the comparative aspect of his' 
research design while examining the association between judicial 
attributes and judicial decision making in a wide variety of 
substantive case. 'areas. The^e methods involved coding the. voting 
of justices (the dependent variable) as "above" or "below" the 
average of their court's support for the claims of various 
classes of litigants to compensate -for the fact that each gro\ip 
of judges heard a completely different set of cases. 

Prom a statistical perspective ,> Schmidhauser ' s and Nagel 's 
pioneering studies' were unexceptional. All of Nagel 's 
demonstrations of the relationships beto/een attributes and voting 
behavior are based on two by tw^o contnigency tables accompanied 
by a statistical test for the significance of .the difference in 
dependent variable proportions. Despite the* problems ^.posed by 
his smaller N, Schmidhauser does present some muitivariate* 
analysis of the relation between Supreme Court justice attributes 
and the dependent decision variables he analyzes. But since the 
multivariate analyses are contingency tables of up to 16 cells (2 
X 2x2x2), they suffer greatly from small and unstable cell 
frequencies . 

Nagel (1963) was responsible, for another innovation in the 
methodology of judicial research during this period. In 
connection with his studies of the relation between judicial 
attrj.butei3 and judicial decision making, he gathered data on the 
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"off-th^-bench" attitudes of his sample of judges. He thus was 
able, for the portion of his sample who returned the survey, to 
operational ize significant attitudinal varibles independently of 
•measures of the , values represented in ' thdir decision, making 
behavior^. These variables were then integrated in a limited 
fashion^ with attributes into analyses of decision making 
.behavior. Since it is this last innovation which was of most 
significance to the development of judicial behavior research, it 
IS unfortunate that Nagel's analysis using the attributes, off- 
the- bench attitudes, _ and decision behaviors was cursory and once 
again limited to simple cross-tabulations (but see Nagel , 1974). 

Finally, Nagel's research bears mentioning because it 
represents the most self conscious effort before 1965 to develop 
and implement a complex research design to guide data gathering 
and analysis. 

Another important methodological contribution during the 
« period of self ccjiiscious judicial behavior studies came from the 
^ work of DanelskiX^nd Murphy with the archived private papers of 
Supreme Court justices. This alternative data source, a staple 
of such traditional judicial research as the judicial biography, 
was innoyatively used by these two scholars to suggest, confirm, 
and infirm behavioral hypotheses concerning the procedures and 
substantive decision making of the U.S. Supreme Court. 
Oanelski's study of the influence of the Chief Justice (1960) and 
of' the appointment of Pierce Butler (1964) used concepts* and 
methods ^ borrowed from Cognate, disciplines (especially 
. transactional analysis in the latter) to expand the arsenal of 
promising approaches to the study of judicial behavior. Murphvy • 
used the papery to provide data in support of the existence /of* 
certain judicial strategies inspired by game theory. Their iise 
of these data no doubt sensitized their colleagues to t^h'e- 
potential of such non-official information for expanding 
knowledge of judicial behavior. 

Outside the development of bloc and scalogram analysis, the 
most significant statistical developments in this period were the 
introduction of factor analysis of judicial decision makingx-n>y 
Schubert (1962), reinterpreting a previous analysis by ^the 
psychologists Thurstone and Degan (1951),- and of multiple 
regression analysis by Bowen (1965) as a technique to allow the 
more .systematic assessment of the relation of judicial'' 
backgrounds, to judicial decisions. ^ The former was critical to 
Schubert's development of the full psychometric model in The 
Judicial Mind ; the latter provided what appeared at the time to 
be the definitive statement concerning the analytical ©roblem it 
addressed. . f ^ • 



IV. . From The Judicial Mind to the Present: Judicial Behavior 
Methodology Matures \^ 
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The Judicial Mind and the Psychometr ic ^ Model * 

(♦ 

The measurement and statistical methodology of The Judicial 
Mind were so complex and/ in full array/ so innovative that they 
overshadowed the book's substantive conclusions.. The latter 
were, after all, presaged in simpler forms in) the article 
literature produced by Schubert (1962af 1962c / 1963d, Ulmer 
(1960, 1962) and Spaeth (1963a, 1963b) prior to the publication 
of The Judicial Mind* 

Taken singly, none of the methods used in The Judicial Mind 
was unusual: even factor analysis had already appeared In 
studies of judicial behavior (Thurstone and Degan; 1951). And 
the iiata analyzed were "standard" judicial votes in non- 
una|rimous cases. But the magnitude of the quantitative analysis 
it reported and the combination of cumulative scaling with "case- 
wise" or "Q-type" factor analysis were unprecedented. In 
addition, its absolute fidelity to the psychometric model and its 
• ultimate use" of Guttmanys circumplex and radex to depict the 
interrelations of the dimensions of liberalism were unique, as 
was its use of the cumulative scales to rotate the factor axes to 
verify that they were measuring dimensions of judicial liberalism 
towards civil liberties and economics policy questions, ' in most 
cases. 

With one exception, ^ the quantitative methods used in The 
Judicial Mind were open and replicable. The exception was the 
procedure for rotating the factor axes so as to maximize the 
correlation between the projections of the justices' positions in 
the factor ^pace on the rotated axes and their ranks on the 
cumulative scales. The rotation was intended to demonstrate the 
.fundamental identity between the content-less abstractions which 
are factors and the content^full curfiulative scales constructed ^ 
within pre-defined substantive universes. Since there existed at 
the time no known mathematical solution to the problem Schubert 
wanted to solve, he devised a. set of ijberative manual procedures 
to provide a satisfactory - appro^timate solution. Some ' years 
later, his student Gow (1979)- reported a mathematical solution 
and a FORTRAN program fpr this problem. In addition, although I j ' 
have not attempted*^^ verify it, it seems likely that current 
factor comparison and confirmatory factor analysis ^procedures 
(see Levine, 1977) are capable of providing satisfactory 
solutions. 

Varlatiion's of The Judicial Mind 's *|)sychometric model of the 
U.S. Supreme Court have appeared in the years since' its 
publication. Only a few have used the full methodology it 
presented; some have suggested alternative analytical approaches 
or techniques within* the general confines of the model. The most 
substantial use of the model was made by Schubert himself in The 
Judicial Mind Revisited , (1974). Substantively, The Judicial Mind 
Revisited was important for its demonstration that the dimensions 
of judicial ideology which characterized the Vinson and early 
Warren courts continued to characterize the latter through its 
temination, and for its revisions of some of the minor and more 
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speculative conclusions of » The Judicial Mind . Methodologically, 
xt was xmportant because it reported an explicit effort by 
Schubert to compare the results of four different methods of 
dimensional analysis: principal components and oblique (oblimin) 
factor analysys, smallest space analysis (a variety of non-metric 
multidimensional scaling — see Kruskal a*nd Wish, 1978), and the 
centroi-d factor analysis used in The Judicial Mind . Although 
Schubert concludes that all thre.e methods used in The Judicial 
Mind Revisited produce results superior to those ofThe centroid 
factor analysis used in The Judicial Mind , with smallest space 
analysis being preferred, it is fortunate for the conduct of 
analysis that his investigation also demonstrated the essential 
convergence of conclusions drawn from the results of any of the 
approaches, eVen the centroid analysis, allowing for differences 
in the time periods covered and data coding of the two studies. 

Other applications inspired by the psychometric model which 
used some form of multidimensional analysis of judicial behavior 
were noteworthy not for the contributions they made to the 
development of quantitative methodology, but for the fact that 
they attempted to study non-American settings. These included 
Schubert's, (see 1969b, 1969c, for examples) and Blackshield' s 
(19 72) studies of the Australian High Court , Danelski's (19 69) 
analysis of the Japanese Supreme Court and, quite recently, 
Robertson's (1982) effort for the British Law Lords. 

More Comparative Judicial ' Behavior Research 

The, studies just cited were among a number of comparative 
analyses in the years following, the publication of The Judicial 
Mind. Studies appeared which reported social background 
analysis and/or cumulative scaling or, other forms of analysis of 
judicial decision making for Japan (Dator, 1967; Kawashima, 
1969),. India (Gabbois, 1969a, 1969b, 1970, 1974), the Philippines 
(Samonte, 1969), Canada^ (Fouts, 1969; Peck, 1969), England 
(Morrisson, 1972), Germany (Kommers, 1969), and Switzerland 
(Morrisson, 196,9). Aside from their us6 of data from non- 
American courts, however, there was nothing methodologically 
innovative*^ ' about' these studies — indeed several of them 
contained minor to significant errors in the applications of the 
quantitative techniqyies they used, no doubt due to their authors' 
well-meaning use of I methods with which they were not especially 
familiar. Their methodological significance lies in the fact 
that they began to provide the broad range of data and 
generalizations which is necessary for the development of a fully 
matured science of judicial behavior. 

Some non-psychometric comparative analyses published since 
1965 have been innovative in their use of quantitative methods. 
Among these are the causal models of the Australian High Court 
reported by Schubert (1969a), the "quasi-experimental" AID 
analysis and causal models of recruitment to the British higher 
judiciary, by Tate (1975a, 1975b), the regression analysis of the 
impact of social background factors on the voting behavior of 
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Philippine "Supreme Court justices by Tate (1972) f and the 
sophisticated analyses of mail survey data from Japanese (Dator^ 
1969) and Austrian and Swiss judges (Wenner^ Wenner, and Flango^ 
1978) and personal interview survey data from Dutch judges (van 
Dunn6 et al. , 1982). To this category also might be added the 
rej)orts by Schubert (1982, 1980, 1977) on his research using 
multiple data sources on South African and Swiss judges, although > 
a hea^^y reliance on multidimensional analytical methods without 
adequate exploration of the data being 'subjected to such analyses 
mars their accessability and utility. 

Enhancement of Foundations , 

* . 

The methodological foundations of judicial behavior research 
represented by bloc analysis, cumulative scaling, game 'theory, 
the analysis of judicial attributes, and judicial surveys, which 
were laid in the period of the development of a self conscious 
judicial behavior movement, were significantly expanded upon in 
the years following the publication of The Judicial Mind . 



Bloc Analysis and Cumulative Scaling 

The procedures for bloc analysis were rigorously 
systematized by Sprague (1968). For better or worse, this mad-e 
it possible to construct replicable bloc analyses even from data 
whose content was not well understood. ^ A similar capability 
became commonplace with the development of computerized 
clustering programs which could use appropriate measures of 
association (such as Yule's.Q or Eta) to isolate clusters of 
"items" which would form cumulative scales after appropriate 
subsequent processing .(see the CORREL and GSCALE programs in the 
(DSIRIS III package, for example). 

The new , bloc analysis techinques were fairly quickly 
integrated into judicial research. The new approach to 
cumulative scaling has not been: analysts have still ordinarily 
identified the items to be scaled on substantive grounds (which 
requires a good prior understanding of the data) before 
submitting them to Guttman scaling programs which then spare the 
• analyst the drudgery -of s^cale .construction. More Significant 
than the above noted deV'elopments, however, has \ been the 
essential replacement of , both bloc analysis and cumulative 
scaling methods by multidimensional scaling or factor analysis 
methods in the work of some scholars (Ducat and Flarigo, 1977; 
Robertson, 1982). After all, both techniques were designed to 
assess • diritensionality in judges ' decisional behavior . 
Multivariate forms of dimensional analysis allow such an 
assessment to be made without making prior, possibly incorrect, 
assumptions as to what that dimensionality is. 



Game Theory 



19 



ERLC 



the nerJvin«hf?}i°''^;^ ^^'"^ ^^^^^^^ approach benefitted from 
the new availability of unique archival data: the do'cket books 

data "on thfif '«irrH"?'°'^'"''°" ^'^^ ^1^^^' who receded 
data on their, and their colleagues' voting in conrerence on 
certiorari (Burton) and initially on the merits? information 
which previously was unavailable. The work of Ulmer (1978^1972) 
and Brenner (1981, 1979 ) allowed game, theory based wirk on 

?r tSe'l^vef of 'tS' ""^--Jl-^ility had pLvLusly' re^^rLteS 
to the level of the court or to not directly verifiable 

iSdr^rSSa? W?"'''^''"? penetrate to the behavior of 

ir^ifir ^""ff-u.^J" ^^^^'^ however, it appeared that with 

greater data availability from private papers and from more 

i??r"?9 9: P^\li- records (see^L work of Slotnrck? 

iv^^I/i, 'u^^^^^^',^^P°*^^^^^^ concerning internal Supreme Court 
procedures became less and less closely tied to formal game 

exoJoUs boJr'"? a good example of a work which 

?n » ? private papers^nd public data, although^ it does so 

in a methodologically pedestFian manner. 

Judicial Attributes and Recruitment ' 

vari«h?pf of the ability of social background 
^^oo?^ ° explain variation in judicial behavior was 

i"":! results plus a host of theoretical objections 

^vn?«n«?° Grossman, 1966 ) to the backgrounds-decisions 

explanatory link explored by the work of Schmidhauser and Nagel 
reviewed above suggested that while backgrounds research for the 
purpose of describing judicial recruitment might be worthwhile 
further efforts to explain judicial .decision making in terna of 
background factors was not-. 

"The critics of this .research were unable -to demonsrate the' 
ov^i °^ theoretically appropriate models 

^ K° . however. Furthermore, . preliminary 

regression baaed research by Ulmer (1973, 1970) and" Tate ( 1972). 
continued to demonstrate moderate empirical explanatory potential 
for judicial attributes as predictors of judicial voting behavior 
l^ao'^. °^ dissent behavior. Recently, Tate's 

(1981) research has demonstrated that such attribute models ca^ 
.in fact be empirically very powerful. 

K 5??^^ .,°^ the^^explanatory power of Tate's attribute models is 
probably due to Careful attention to operationaliisation and 
measurement of ±he independent attribute variables p?evioSs 
research had foukd to be linked to judicial decision making and 
to sensitive use of multiple regreaaion analysis. .His results 
suggest that similar improvements in the data ' anaO^sls 
methodology of judicial behavior research on other topics would 
be well worth making. The work of Gibson (1978, 1981) is a - 
further example of the potential payoffs of methodological care. 

Expansion of Survey Research 
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The major methodological ^development t:oncerning ' data 
generation procedures which occured in the wake of The . Judicial 
Mind was the very substantial expansion of survey research on 
judges. Proceeding from a variety of theoretical postures / 
pioneering scholars conducted personal interviews with judges 
from ^several ^ levels within a single state (Henderson and 
Sinclair f 1965)/ supreme court justices in four states (see 
Click, L970; 1971), federal district judges (see Cook, X971; Cax^p 
and Wheeler, 19 72)^ and federal Circuit Courts of Appeal judgtes 
(Hpward, 1981). While only an enterprising journalist (Grey, 
1967, 1968) was able to secure interviews with (8 of the 9), 
justices of the U.S. Supreme Court, political scientists studying 
foreign judiciaries were more successful. Kommers' (1969) and 
Morrison (1969) reported interviewing some members of the highest 
courts of West Germany and Switzerland, although it does not: 
appear that their ' interviews were systematic. More 
significantly, systematic interview data was gathered by Schubert 
(1982, 1980, 1977) (supreme court judges in South Africa and 
Switzerland), van Dunn&, va* Koppen, and ten .Kate (1981) (Dutch 
trial judges), and Paterson (1982) (British Law Lords). 

Other scholars followed Nagel's lead and used mail surveys 
to gather data on judges. Examples which represent the analysis 
of mail survey data from a variety of- types of American judges 
include the .studies of Hdnderson and Sincliar ( 1965 ) ^ .(Texas 
judges). Cook (1982) (woijien state trial judges with, a matched 
sample of males), Marquardt ^nd Wheat (1982) (administrative law 
judges, using data from a supervisory agency survey). Wold (1974) 
(state supreme court justices), Hogarth (1971, 197?) (Canadian 
magistrates), and Ryan et al^ (1981) (a large national sample of 
trial judges). Studies arfalyzing mail surveys. from samples of 
non-American judges include Dator (1967), and Wennjer, Wenner, and 
Flango (1978) . . 

Surveys "of judges had become so c4nunon by the. la-te 1970*s 
that it is hopeless to list them all. Most scholars who have 
recently used the survey method appear to have done so 
competently. Nevertheless, there has been little attentipn paid 
to the problems of survey research with judiciaries since the 
middle 19.60 's. Perhaps this is a sign of maturity in the use of 
the method, but it seems more likely that it is the result of the 
pressure of limited publication space: in this and other areas 
of judicial behavior research discussions of method are excised 
in favor of reports of substantive findings to save space. 



Introduction of Content Analysis 

s 

Limited and not necessarily rigorous content analysis of 
documentary materials is necessary to the interpretation of 
blocs, scales, and factors^ in the psychometric mo^el and lines af 
inquiry related to it. But prior'to the mid--*f60's, there had 
be^n no rigorous applications of content analysis in judicial 
behavior research. Danelski (1966, 1970)^and Schubert (1965') 
changed that by systemati easily coding the content of judicial 
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"speeches (Danelski) and opinions (Schubert)' to operationalize 
i^hnK^??f ^?Q^?f "^-^ relevant to judicial decision maJcing/ 
Schubert s (1972) assessment of cont-ent analysis suggests that it 
IS more trouble than it 'is worth, given its ,data yield; But tl,e 
need for a method of determining independent, measles of judicial 
values (and other concepts) is sufficiently griat ' to, suggest 
otherwise. • \ 

There are two barrios to the more extensive use of content 
analysis in the study tof judicial behavior. The first is 
technological:- despite the .revolution in computer technology of 
the last decade, truly aWequate computerized cc^nt'fent analysis 
programs still, do not existir(see Krippendorf, 1980 ). There are 
quite useful programs which could remove much of the drudgery of 
manual content analysis (see MadrdTi, 1982 for a di&cussion and 
examples), but they are apparently not widely available to social 
saientists. Furthermore,^ use of even such programs as are 
available runs squarely into the second obstacle: the necessity 
of converting the documentary data to be analyzed into a machine 
readable form. • In the .future, the spread of electronic 
pubXishing. W1I4 remo\/e this problem: all published material will 
liXely exigt in machine readable form at some 'point, and it will 
be a relatively simple matter to manipulate such material " into 
appropriate formats for computerized content analysis. For the 
present and possibly also for the future, depending upon the 
'^^^^^i?*^^®"^ which are placed upon scholarly analysis Of machine 
readable copyrighted materials, there will be costs, sometimes 
substantial- ones, associated with the conversion (oF^rinted 
materials into an electronic form. 




Statistical Developments 

If the development of computerized approaches to content 
analysis has continued to lag, the development of powerful, 
flexible computerized data manipulation and statistical analysis 
techniques has not. This has meant that it has become easier and 
easier ^ for scholars of judicial behavior to apply even the most 
mathematically complex methods to their data. Under these 
circumstances, it would be surprising if "statistical overkill" 
' were not characteristic of -at least some research. In fact, 
_there have been examples of studies which have used complex 
multivariate procedures prematurely, i.e., without sufficient 
exploration of the data by the "simpler techniques which reveal 
the data's essential characteristics to an analyst. 

More-- common .than statistical over.kill, however, has been 
'statistical underkill," the failure to use , appropriate 
multivariate methods/when analytical purposea^ould have been' 
much advanced by theYr use. For example, ^al>3±^ continue to 
use bloc analysis when multidimensional analysis tecKiH£[ues would 
be more appropriate. Schubert's (1972: 118-19) judgement was 
that there was no justif i<?ation even for systematizing bloc 
analysis procedures^ since they should be totally abandoned in 
.favor of more powerful methods. I would not go so f^V^n^oc 
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analysis may still be useful for certain purposes, ' including 
pedagogical ones, and there is a strong likelihood that- 
substantive conclusions drawn from bloc analysis will converge 
with 'those drawn from factor analysis or —'multidimensional 
scaling. But I would agree'that serious contemporary research 
should rarely use bloc analysis. 

Similarly, studies continue to he published which use only, 
tabular and usually bivariate analysis with at most weak and 
sometimes not appropriate measures of association and 
significance, when regression, or discriminant analysis (perhaps 
using dummy variables) would allow drawing much more reliable and 
valid conclusions from the 'data. Regression analyses which are 
published sometimes still unthinkingly adopt the linear additive • 
model when it is possible that a non-linear function would be 
more appropriate or when variable interactions shoul'd be included 
in the models. 

Identifying ' studies I 'consider guilty of statistical 
overkill or underkill is less useful than reviewing some which 
have provided examples of appropriate 'statistical innovation. 
Some of these havfe already been cited in the discussions. of the 
psychometric model^ comparative judicial behavior, and judicial 
attribute analysis above, ^ and will not .^e reviewed here. 
Noteworthy others include Ulmer's applications of discriminant 
analysis to the es^pianation of the decision making behavior of 
Supreme Court justices as individuals (1969, 1974) and as a gr6up 
(1970), and Gibson's (1978., 1981) use of factor analysis aS a 
data reduction tool, multiple regression with interaction 
variables, and path analysis to investigate the effects of 
judges* role orientations and attitudes and of their sense of 
self esteem on their decisions. 

Much statistical development has occured in a large series 
of sentencing studies. While these often fo^ius relatively little 
on explaining the decisions of the judges passing sentence in 
comparison with the defendant being sentenced, the better studies 
do have a potential contribution to make to the explanation of 
judicial behavior. Spofin, Gruhl, and Welch •(1981-82), for 
example, use multiple regression and path analysis well tp 
explain variation in sentence ^verity -and incarceration rates, 
even though the levels of explanation achieved are not especially 
high (as measured by R ) . A sound and interesting, etlthough not 
ultimately very useful, attempt to Use canonical correlation as a 
means of solving the problem of the essential duality of 
sentencing (incarceration vs. probation; sentence length, if 
incarcerated) is reported by McDavid ^ and Stipak (1981-82). 
Finally, one might cite the "best case analysis" by Kritzer 
(1978) which manipulates data skillfully in an effort to assess 
the political cdrrelat^s of the behavior of district judges. 



Longti tudinal Studies 

In the 1970 's some analytical attention began to turn to 
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longtitudinal apprgaches to research on individual judicial 
K^'S^^K Judicial Mind and The Judicial Mind 

Revisited had had^ongtitudinal^rmension~ they ha d 
attempted to develop, ^models of change in the behavior of 




Xongtitudinal analysis is important because it represents a 
new research design in studies of individual judicial behavior 
and introduces a new- class of quantitative methods, those 
associated with econometric time series analysis. The work done 
by Ulmer so far points clearly toward the likely utility of 
pooled cross sectional time series analysis," a technique which 
IS now becoming available' to analysts (iij the SAS computer 
program package, for example), for sensitive modelling of the 
decision- making of judges and justices across their careers. 

V. Progress and Poverty: Judicial Behavior Research Now and in 
the Future 

The Current State of Methb'doloqy ' ' , 
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One is tempted to make a "good news, bad news" analysis of 
the .current state of methodology in judicial behavior research. 
The good onews is that judicial behavior research flourishes, 
oabounding with a greateJ^ viariety of data sources, exhibiting 
more serious attention to operational ization and measurement, and 
using a greater variety of statistical methods and measures than 
ever before, even beginning to take seriously questions of 
research design, including the relevance of comparative work. 
The bad news is that ,^uch judicial behavior research is 
metpoa«a.ogically sub-optimal. 

I ' .use the term "sub-optimal" deliberately. It implies not 
that the research is "wrong" or "flawed" because of improper 
methodology, but that it fails to use methodologies which would 
mine, . the . full worth from the hard-won data of the study. 
Frequently, this occurs because the analyst apparently follows by 
rote the approach taken by one or more of the pioneers of the 
tieid in a^simiXar situation, perhaps not realizing that th6 work 
of pioneers is almost always rough due to conditions on the 
frontier. Less defensibly, it occurs because the analyst is 
apparentLy afraid or ignorant of multivariate methodology which 
reqiiires the use of computer programs which produce 
mathematically imposing printouts or, vice versa, tries to use 
multivariate results prematurely to make points which can only be 
made clearly, through the use of less powerful analytical 
techniques. And, understandably, it" occurs because limitations 
on research resources prevent the analyst from securing all the 
data which theory suggests would be useful. ' 
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What can be dong to remedy these problems? **'Fot the problem 
of limited resources, it'iriight appear that thSre is no solution 
other than iacreased finding for social research. But there is 
one step which could help immensely and which would require only 
limited resources, ' and that is more- systematic archiving of 
judicial ^ data. Some strides have beisn' made in this area since 
Nagel, Schmidhaus^r , and Schubert first archiived their ^data sets 
with^ the ICPSR: the Consortium's archives now hold at least 18. 
studies which deal more or less directly with the judiciary and 
over 40 on the criminal justice system. But there remain dozens, 
perhaps hundreds, of data sets which are produced to support 
research etnd, once used, lie fallow in filing cabinet drawers or 
on forgptten computer tapes until they, are finally lost. It may 
be that not all these data set^ deserve to be archived, but it is 
almost certain that few of them are fully exploited by the 
scholars who initialy collect th^m and many of them, could provide 
the supplemental data, an analyst imight need to fully exploit his 
or her own data by more completely operationalizing central 
hypotheses . - ' " ' ^ * . ^ 

Let nie use my own case as an example, both of the failure to 
archive useful data and of the utility of usin^ data archived by 
others. One of the .most interesting data collections for which I 
have beenr responsible is a set of social background and career 
data onr the judges of the ftr-itish higher judiciary from 1876- 
1972. These data were used in two analyses (Tate^ 1975a, i975b) 
and^ may still support additional research on my part on British 
judicial recruitment. The data were requested and have been 
archived by the Social Science t)a.ta Archive at the University of 
Essex, and have been distributed privat.ely t6 a number of other 
scholars who have .requested them. They have not been, so far, 
archived with ICPSR through rxa fault of that organization — I 
have made no serious effort to get them archived there. Given 
the encouraging recent developments in the study of British 
judicial behavior (PatersoH/ 1982; Robertson, 1982), it- is likely 
that these data would become of increasingly great utility, if 
archived. 

My research (Tate, 1981) on the impact • of judicial 
attributes on the civil liberties and economics* voting behavior 
of U.S. Supreme Courj: justices is an example of the kinds of uses 
which can be made of multiple sets of archived data. This study 
used data originally collected by Schn\Mhauser and Schubert as 
^v^ll as additional data • generated by^ my colleague .Richard 
Jo\nston arid Andrew Van Esso through the courtesy of Johp . Ryan 
and . the American Judicature Society. Had the Schmidhauger and 
Schubert data not been archived by ICPSR, the^study would not 
have been possible, but it is equally important to note that the 
archived data were supplemented by data generated by my 
colleagues. This analysis of data from multiple sources, not 
simply secondary analysis of archived data from a single source, 
illustrates what is probably the most significant reason for 
archiving data on judicial behavior*. 

\* 
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The contrast between the. archi'ving of judicial and 
legislative data ,is instructive,^ Since its early days ^ the 
Consortiuip. ha§ routinely coded a'nd arcliived Congressional roll 
call voting .data ^ butnot U.S. Supreme Court voting data> despite 
the, significance of the latter in judicial behavior research. . tAs 
a consequence / .updating of that data- set depends upon the. 
initiative of enterprising scholars of judicial behavior. At 
bestf^ this ^jieans that at ' least some spholarlV effort is wasted 
through unliecessary du^licat|^dh., , At worsts it means that the 
timeliness of the Supreme Court voting data set suffers and that 
^ the discipline is subjected to?-^ idiosyncratic and non-comparable 
research . • * " r . ^ . 

< I do not wish, to^ overstress the signif xca^nce of'^^U.S. Supreme 
Court voting data; there ^ are surely other worthy candidates for 
routine coding ^nd archiving. But in the absence of such 
routinizationf judicial behavior scholars would do well to at 
least increase communication concerning ' who is coding what 
official records and when. 

What/ can be done ^ to improve -quantitative 'analysis in 
judicial behavior? The heed now is not for every ' scholar to 
s,earch out the ^newest and most esoteric method'*"fe?>^ply to his or" 
her data in order to demonstrate his or her superior 
'methodoldgicar skills. Rather^ the. need i^ constantly to 
confront: data analysis with theory / to ask insistently "Have the 
centiral theoretical concepts been measured as rigorously and 
imaglhatively as possible?" ^Have/ multiple or alternative 
operationalizations of the^se concepts been tried?"/ "Have the 
levels of measurement-^ of the iadicators been maximized-?"/ "Have 
potentially 'significant ' interactions been considered and 
raodej^led?"/ "Have appropriate multivariate methods been used to^ 
test hypotheses which in fact require testing tihe Impact of 
multiple vat^iables (as most do)?"/ "Can alternative data which 
would allow more complete testing of my hypotheses be secured 
with reasonable effort?" 

Achieving the kind of analyst self 7Consciousness described 
above will not be pasy. ' Few scholars are immune to the urge to 
rush ■ to print once* data has been gathered ancj data analysis 
initially . completed. What will encourage them to be more sej.f- 
conscioTO?r (First/ better training. MethodologicaT nflivete 
invariably accompanies methodological ignorance. I am tempted to 
say that the problem of training judicial behavior scholars is 
moKe significant than for other kinds of, political scientists. 
AfJ:er all/ most contemporary judicial belfevioralists . still have 
to live an alternsfte J.ife as^ ^traditional public law teacherS/ 
even i*f they do hot engage irt public law research.. Becoming and 
keeping current in public law may interfere with becoming 
methodologically -competent. But in fact this probleih/ while- 
serious / is , certainly not unique. Analogous situations surely 
confront political theopistS/ who must teach the classics while 
they do research in. formal theory/ and comparative politics 
scholars / who find it necessary to txy to maintain an area 
studies* expertise while becoming competent in croas national 



quantitative analysis. Furthermore, it has not deterred many of 
the pioneers of • judicial behavior from steadily expanding %their 
methdological sophistication and expertise* 

The /records of the pioneers surely suggests that formal 
methodological training, while •helpful , is hardly essential. 
Willing scholars car^ become -as •m^hodologically sophisticated as 
their^ ^research demands, or else thay can • collaborate with 
colleagues whose skills balance their own (see Ducat and Flango, 
1977, * for a good example). Training needs to be supplemented by 
the establishment of methodological standards by journals which 
publish judicial' behavior research. These standards might try to 
operationalize the questions listed above. Editors might insist 
that authors, not just reviewer?, answer the questions candidly, 
either in their articles or in statements submitted to document 
acccepted articles, but not to be^ published, given scarce journal 
space. Such statements might contain statistical and 
* methodologica l detai ls whj.ch could not be published, but which 
^ could^ be made'aivaiiable ^t cost of reproduction to interested . 
scholars. 

These comments should not give the itfipressi'on that the sub- 
, optimality of the- m^hodology of judicial behavior research^ is' 
tfnique to the field. To the contrary, I am sure that similar x 
problems exist in all areas of political behavior research. 

Methodology ^ i n Future Research 

In closing, it maybe useful to speculate concerning possible 
trends in judicial behavior methodology in the near future. Todo 
so, I shall return to the initial classification of aspects of 
methodology made in section II above. 

Research Design and Comparative Method 

m 

First, it seems clear that the broadening of the scope of 
judicial 'behavior research through comparative analysis of non- 
American judges will continue,, indeed accelerate. The beginnings 

^ which were made with Schubert and Danelski's encouragement in 
1969 may have 'been premature, since there has been only sporadic 
judicial behavior research on non-American judges since. But 
recently, the pace has picked up . (Schubert ,1982 , 1980, 1977; 
Wenner, Wenrier, and Flango,' 1978; van Durfnfe, van Koppen, and ten ^ 
Kate, 1981; Paterson, 1982; Robertson, 1982), and there is reason 
to expect it to continue to increase as more non-American 
scholars apply the assumptions and methods of judicial behavior 

^researcjh to their own judiciaries. 

Second, judicial behavior studies will incresingly use truly s 
comparative ^(i.e., "cro^s-stata/circuit/di.strict/city" ) research 
\^ designs. This trend has been under way sinc^ the late 1960 's, 
but, as it continues, it will reflect more conscious attention to 
designing comparative components into' research.^ 

t 
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Third, judicial behavior research will become more 
longitudinal. what Ulmer has done for some U.S. Supreme Court 
justices can and should be done for more, and for the justices of 
many other courts. As such studies are done, they will 
incorporate more contextual data from the judges environment in 
efforts to develop fuller and itipre satisfying explanations of 
judicial decision making. 

Fourth, judicial behavior research will be characterized by 
mor^ experimental and robust quasi-experimental research designs. 
Just as judicial behavior scholars found it possible to penetrate 
the 'purple curtain" with the survey instrument, they will find 
occasions to employ more nearly experimental designs to assist 
them in explaining judicial decision making, in this connection, 
1^ the urgings of Schubert are taken seriously, judicial behavior 
research designs will include truly biological variables, as well 
as those such as sex (see Cook, 1982) and age which are 
biocultural, not just biological. 

Data Sources and Generation Procedures • 

Judicial behavior scholars will continue to expand^rhe data 
sources and generation procedures they use. One shoul'd expect to 
see still more imaginative use of official documents and archival 
materials from a greater variety of judicial settings, and more 
and more significant survey data. . In addition, there should be 
greater use of systematic unobtrusive observation, participant 
observation, and even clinical, observation in service of 
experimental and quasi -experimental research designs. 

Continued developments in 'computer, especially 
microcomputer, technology will'finally bring content analysis of 
documentary materials into its own, with potentially enormous 
benefit for judicial behavior and even for traditional doctrinal 
research on judicial decision making. \ 

' < 

Operationarization and Measurement 

The availability of new and new kinds of data from "the 
procedures described in the preceeding paragraph will encourage 
better "operatxonalization and measurement, as will the spread of 
knowledge about the -measurement techniques being developed in a 
variety of social science disciplines. Improvements will also 
-occur as the contributions of some of the measurement conscious 
scholars discussed above become more widely known. 



Statistical Methods and Measures 

The most significant progress in this area of * methodology 
will occur through more effective use of known, standard 
statistical methods such as regression analysis, rather than 
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through the use of newer or less generally applicable methods. 
Nevertheless/ one should expect to see greater use of any number 
of kinds of multidimensional analysis techniques / including many 
varieties of factor and multidimensional scaling analysis; of 
various kinds of "maximum likelihood" statistics/ both those 
analogous to regression and to factor analysis; of time series 
and related longtitudinal analysis methods / including ARIMA 
models and pooled cross sectional time series analysis; and of 
categorical data analysis techniq^s such as log linear 
modelling. 

As in the A|^t/ many of these- will be used for the novelty 
or "bragging rigiit^" associated with being one of the first 
users. Furthermore/ there is a strong likelihood /that Initial 
uses will be more .cbnfusing than enlightening. With experience/ 
however/ scholars will discover appropriate uses for these 
methods and their readers will come to ^under stand when they do 
and do not have utility. 

The Complexity of Judicial Behavior Research 

Gibson ( infra ) has noted the apparent fragmentation which 
characterizes contemporary judicial research. Certainly 
developments in methodology have contributed to this as 
proponents of one methodological approach have insistently done 
things their way regardless of the work of others or the 
availability of alternate approaches. Nevertheless/ such 
increasing complexity and fragmentation have ordinarily been 
associated with the progress of scientific** .idlsnciplines. From 
this perspective/ judicial behavior ha^ come far toward 
fulfilling the hopes of its founders. 
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